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(57) Abstract 

A data warehouse system and method. The data warehouse includes a memory and a processor. The memory includes a database 
having a plurality of data entries. The processw includes a cache and an override engine, wherein the cache includes a subset of the 
plurality of data entries and whciein the override engine extracts data from the cache for viewing by a user, modifies the data in response 
lo one or more user commands and saves the user commands to a file for later application to the database. 
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SYSTEM AND METHOD FOR UPDATING A MULTI-DIMENSIONAL DATABASE 



Rark proMP ri nf thp Invention 

Field of the Invention 

The present invention relates generally to database management, and more 
particularly to a system and method for manipulating facts in multi-dimensional databases. 

Background Information 

Business decision-makers operating in today's rapidly changing business environment 
need answers to a host of questions that directly impact their ability to compete in the 
marketplace. To manage and use information competitively, many companies are 
establishing decision support systems built aroimd a data warehouse. A data warehouse 
stores a company's operational and historical data in an integrated relational database for 
decision support applications, business data access and reporting. Decision support systems^ 
access such databases to analyze and summarize corporate performance. 

Data warehouses employ relational database management systems that use a language 
such as SQL to retrieve rows and columns of nxuneric data. The systems may also permit 
access to textual files such as documents. Data may be accessed directly via user-generated 
SQL commands, or indirectly, via an interface which generates the desired SQL commands. 
AppUcations such as Business Objects from Business Objects S.A., France, - 
(httpi/Awsw.busihessobjects.com), Forest and Trees from Platinum Technology Inc., 
Oakbrook Terrace, XL, (http://wvvrw^^>latihiu^ Pilot's Li^tship from Pilot Software 

Inc:, C^bridge; Iv^ 

which usej^rpwse windows under tji^ TOntroI pf end-iise^ to generate tl^e SQL code needed 
to analyze the data in the data warehouse. / ; 

This approach, however, runs into significant performance problems associated with 
PC and network limitations. !Q^OTe;s generated by inexperienced users can dominate and 
crash the database, or cause excessive! network congestion. In' addition, there is no 
mechanism for shifting large processes so they execute during off-peak hours. 

QLAP (OnLine Analytic Processing) technology, also called multidimensional 
analysis can also be used to access tiie;data warehouse rplationaV database. Multidimensional 
analysis systems have been available for over 15 years, first on mainfiiunes and then on 
chent/servers. Under multidimensional analysis, data is divided into the dimensions and facts 
needed to manage the business. Diinensions for marketing applications may include 
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products, markets, distribution channels and time pOTods. The dimensions are used to 
reference specific points in a database. What that point represents is called a fact. As 
examples, units sold, revenue, and price; are all fstcts. Dimensdo^^^ are further described by 
attributes, such as size, flavor, location or fiscal year. . Attributes also describe hierarchies 

5 within a dimension,, even overlapping and inponsistrat W These hierarchies 

detemiine the vqtical relationships \vij^ For exanjple, in a Period dimension, 

a standard hierarchy is year -> quarter -> nipiith -> ^?ek -> d^y. By defining these 
hierarchies, it becomes possible for OLAP ^plicationsHo shift their *view' up 

or down a hierarchy. This is commonly referred to as, •drilling' within this application space, 

10 an example of this would be shifti^i^ an a^ P view the 

iiidividual quarter's munbers^ J r:K ^ i : : i L; ^ : vf ;^- i-u ; ' 

Multidimensional analysis allows^ users; to select, sm^ calculate, format and 
report by dimensions and by attpbutes \yitlm to support virtually 

any timerseries decision support ^qpplicptipn m^^ forecasting and 

15 budgeting.,; -. ^. .. v-.-;::; ;i^^r.h^n y.::.;b b:,ni\,: - . -v,^; . ■ . . . ^ 

To be usefiil, decision support systems must support analysis based not only on 
historical data but also oaprpjections for fiiture aqjiyitips. For instance, marketing may 
project s^es for the next thr<ee months, Th^e figures j^^^ then be introduced into a model 
used to tune manufacturing output oyer tiiat pOTod oC timj?. To date, such^alysis has been 

20 performed usmg multi-dimensional databases having %ed locations, ^^^at is needed is a 

system and method of extracting and mo4ifyTpg fr<>^n existing datable which 

can be applied to a relational database m ord9r to free ihp organization fironi the space 
liinitations of multidiniensipnal databas^^^^ 

method capable of creating reports not only based on existing information but also on . 
25 prcjectiom of" future activities,;, - ^ z*^ ; W ^ > :v . , 

The present invention is a data warehouse system and meAo^^^ 
includes a memory and a processor. The memory includes a database haying a pluraHty of 
data entries. The processor; include^ a cache and an override engine, wherein the cache 
30 includes.a subset of the plurality of data entries and wherein the oveiride engine extracts data 
from the cache for viewing by a user, rnodifies the data in response to one or more user 
commands and saves the user commands to a file for later application to the database. 
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Accoixling to another aspect of the present invention, a data warehouse is described. 
The data warehouse includes a plurality of workstations connected to a memory by a server. 
The memory includes a database having a plurality 6f ^ta entries. The server includes a 
cache anil an ovorride en^e, wherein the sefv^ operates vn response to user commaiids to 
5 store a sibset of tfie plurality' of diata ratn^ m wherein the override engine extracts 

data j&Dm the cache aii^ senHs the d^la exhabted £rom the cache to the client workstation for 
viewing by a user, modifies' the data m n^ponsd to dne or rnf6re of the user 
tlie user coirunands to a' file and ope^^ ui irespo^ to modifies the 

database based on tfie usCT<^ 
10 " Acbording to yet method of re?pbrting data 

fiom a data warehouse is described in which the steps are providiiig a server ^d a memory 
device, stoniig a datai^ includes a plurality of 

data ehtnes; extracting'a siili^i: bftlata ei^ databa&e; stbrihig the- ^bset of data ;|r 

entries oh tiieser^ modifyii^ttie Ssa^e^ fee servdr iri^espbiis^ to user ' 

1 5 commands, reading data fix>m the modified data entries stored on the server and displaying - 
the data to the user. ^ " . . ..^ ^ . .. . ^ . , . . o 

According to yet anotheir aspectof tile p^^^ method of forecasting 

based on data in a daii wjarehoiisfe i* de^ providing a server and a 

m^nory device, the database includes a^- 

20 plurality of datientrie^^ extracting a' subset of dita ratn firom the database, storing the 

sirf>set of e^tri^s on the servei", itio3ii>ai^'t^^ on the server in response 

to user corammds, storing tii^ iiser ctomriia^^ data ifrom the modified data entries 

stored on &e server, displaying the dka to the uiser mdtaddifying the database based on the 
stored user corhihan'ds. ' -m'..-. , 

25 According to yet another aspect of the present invention, a riiethod of increasing the 

speed in which changes to a relational database are reflected back to the \iser is described in 
which the steps are extracting a subset of data frbm' the database, wherein the step of 
extiactiiig includes the steps of displaying a re^reseiitatioii of the subset of data to the user 
arid storing the sublet of data iii a cache, receiving a data modification coriuriand, storing the 
30 data modification command in a'filb and applying the data modification command against the 
subset of data stored in the cabhe; wliereih the step of applying the data modification 
conmiand includes the step 6f ihodiiyiiig the subset of data to reflect application of the data 
modification comrnand. ' - 
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Br^ftf nftscriptf'^ri r>f thft T>rawinps 
, In the drawings, where like numerals-refCT^^^ 

views,- . , , ., .."!: a 7- r- . : - 

Figure 1 shows a data warehouse decision OTppQrt sy^em according to the pre^t 

invention^ ^ . . >>: ■ U <-.q r ,..„■> v,.... 

Figure Z shows a more detailed implepirait^tipn of the data warehouse decision 

support system of Figure I;-... ■■: r- vr; b./tGau-.K-:; .f h-v;: i'': sn r-,, ,.; ;-. - 

Figure 3 illustrates a star schema mplmra^s^Jipn^^^^ acc<»ding to the 

present inventiqn;- . : a-i,- , -.:.,<t ; v-'-s i)'; vnii:vj.Mi-/ 0; !-;t t-'-i v ■ 

.Figure|4 is a more detailed de,scriptjGn of a dat^i warehouse according to the present 

invention;-,, > :,.;;^;-:,,v;r . ^ tr.f-v sinTl.:^^^ li-.f; v- v. ; ^ • . 

Figiffe.5 shows an aftenijate i^boiU^ 

system according to th'3 present invOTtionjiJ^^ . , o 

Figures 6a and 6b illustrate distribution^ of ?td^ across ,leypl^ in an imloc^ed 

andalockedsystem, respfictivelyv ^ ;r 

, PfSfTlptiiftTl rtf tlift Th-effTred/Rmbodiments , _ 

In the following detailed description^of thej>ref«Ted embodinaents, referraiM! is made 
to the accompanying drawings which form a part,h?i;^f, and in which^is shown Isy way of 
illustration specific embodiments in^hich,% invei^tipn inaybe practic^. It is tp l^ 
20 understood that other embodiments may ^be, utilized amd structural changes may be made 
without departing fi-om the scope of the presOTt myoi ; .< ■ . 

Figure 1 illustrates a computer system 10 having ^ enha^iced c^adty to ejrtiact and 
modify data stored in a database. Coniputer syst^ 4 0 includes a ip i;^jCoiin^taI to a 
processor 1 4. Processor 14 includes a cacji^ ,16, a^ r^orting tool 19 and an override engine 
25 18. Memory 12 is used to store a database 20. Database 20 includes a plurality of data 
entries26. . ; • . ■,. . ■, ■ 

. . In one embodimrat, processor 14 also includes an instruction application process 34 
for permanently applying the modifica.tions made v*athin override engine 18 to, data entries 26 
stored in database 2Q. In one such.embodiment, database ?Q is a d^ta warehouse and override 
30 engine 1 8 is implemented as a multidimeiMdonal data entry/edit ^ftware engine process 
which supports the creation and adjustment of data points in the data warehpuse. 
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In one embodiment, as is shown in Figure 2, cache 16 is implemented as a persistent 
cache which stores a subset of the plurality of data entries 26. In one such embodiment, 
override engine 1 8 extracts data fii)m the jiersistbht cache for viewing by a user, modifies the 
data in response to one or more user commands and saves the user commands to an 

5 instruction file 30 for latCT ajppUcia^ ^ "" 

A user sends commands to processor 14 over command interface 22 and receives 
reports based on data rat^ firom' database 20 tn^cugliirepDrl interface 24. In one 
embodiment, as is shown in Figure 2, command interface 22 and report interface 24 are 
impiem^ted as a single graphicial u&ct iirit^ici^ (GU^ ' r ■ . 

10 User action within system 10 is basically an adjustment process of data representing 

futiu-e periods in time^ fii oiio idm^ 

systems and stored in the database 20. An example of this would be a statistically created 
forecast of Mm^e Volume^ A User c£^ri^tfi^^'^^c^V€^^ the Valued (ie., iiacts) that 

they consider to be inaccurate. The combination' of tK^ adji^ with the OLAP. 

15 ability to present the data in \irtuiaiiy business professionals to 

review data within a familiar business context and use their knowledge of the business to 
refine the data warehouse values. 

As with a standard reiad only OLi^ f epbrtiri 10 allows end users and 

administrators to define repbrts'to orjganiie «iHd preseitt data, fhese reports are crfeated by ^1 
20 selecting the desired dWensionid id^^^ that contain the data needed 

to support the decisibri W plaiiiiirig process: >^ m si report may contain facts such 

as 'Annual Plm/ 'Statistic4l' Forecast,' ahd 'E^ with dunensions 'Central 

Region,' 'Com Syrup/ and 'May 98; Jiiri6^8;aftdJu^^ 

All the data the user sees idii &e r^ort i^ stoired in a virork file tha wai^ created for that 
25 adjii^'ent <iycle. Tiie pfebes^ a&iiiustratc^^ file for each adjustment cycle. 

Fof ejtample, iii a mbrithly planning cytle ther6 Wcr^ld bie one work file for the May 1 996 
cycle, one for the June 1996 cycle;;^aridi50 oh: - • - ^^'^^^-^^'--''^ ^ i . ' 

When the user adjusts a fact value, the information is saved in a report file, not the 
work file. In the embodiment, when the final adjustment of the fact is completed, the process 
30 admiriistratorupdktes the database with t^^ 

As an example, the following chart smnmaiizes a typical adjustment cycle. In this 
case, a cycle used to create a comerisus tactical foreciastis shown. The chart^dentifies the 
required input, the steps, arid the results. ^ 
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1— — — 

Override Proems: Tactical Forecast Cycle 


input to tne Cycle * 


otepf 01 ulc ^ycie 




statistical lorecast aata. 


1 ^ users create reppris vising 




lacncai wonc me. 


mici 5 allU UlIllCU&lU****^ ^ ■ 


r"; ' ■ * ■ ■ ^ . 


tacucai laci. 


^V\\fK\r u//M-lr r%vfJ\f Jinfllt f\\^ 

1 uey wpoL oniy^ iriiii H'y, ... 






assigned produpts/ 






marKeis/penocis,^ ? „ : ^.^^^ ■ . 




. , '.. ■■ 'V. 


2, Users adjust Jast mpnth^ 


Tactical forecast 




- . . t&cticai lact lo creaie uusi , , ^ ^ ^ , 


iCLH/l lo xKJx vlUlCJlo lU 




^ , - rnontn s lacncair x^jx^^y* . . „^ 


f 




They save the forecast ^ . . 
amounts, and distribute „ 


change. 




the forec?ste;f^r,reyiew.^ , , 
3. Users make final chM 


Final version of tactical 




; and save Ae^taw^fic^J^o^ . 


forecasts. 




; . ibrecasfs, ^ , , ^ 




Report files, tactical 


4. Forecasting administrator 


Management work file. 


work file. , . 


7 Completes the cyole. 


nianageihent fact 



f ■' ' ' " i 



As an additional example in, which the oyerride function is used in a forecasting 
application, the following chart sunmiarizes; a suhj^^ forecasting cycle. It 

10 identifies the required input, the steps, aid ttie res^^s^^j . . j, i 
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Override Process: ManagenQient Forecast Cycle 


Input to the Cycle 


Steps of the Cycle 


Results of the Cycle 


Management wbric file. 


1' Users cS«ifc reports ii^ 


Preluhiriary reports. 


management fact. 


filters and diineiisions.* 






Tfiey work only With the 






assigned products^ ' 






markets/periods: 






2: lisers a^IjtLsTi^^^ 


Reports for others to 


: 


' faianagem^mtfact tb * 


review, work with, and 




' ' create this '''^ 
management fbtecaist. ' ' 


change. 




'Kiey sav^'ftirMk"'-^-' 
amounts,' ^difistril^^ 


t 




the forecast'&r revie^^^ ' 


I 




3/' Users make ^ai changes ^ 


Final version of 




and save the management 


management forecasts. . 




forecast. 




Report fileSj 


4. Forecasting^iiminisbafer .^^ i 


Final business fact. 


management work file. 


completes the cycle. 





At the fend of ea6h cycle',^ the pk>bess adihim the data for the next 

cycle, this involves s^ving%e data to tfte the work file for the next 

cycle, and setting up the single adjiistable iactfor tha^^ adjustable fact is the fact 

that can be changed or overridden. 

For example, at the end of the tactical forecast cycle, the Forecasting administrator 
updates the database with the consensus tactical forecast and sets up the next planning cycle, 
in this case a consensus management forecast. The following chart lists the steps. 
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Input to the Cycle 


Steps taken by the Forecasting 
administrator . ^ 


Results of the Cycle 


Work file for the cycle 
(tactical woik file, and 
managem^t work file) , 


1 . Appends allreport> files, ^ ; ^ , 
fiomaU users. , acv \^ ? 

O r^nmmitQ tflfi Hata to Hie 

Forecasting^ datahas<e. , , 
3, Sets the fiact to be , ^ ^ 
adjusted in the next 
forecasting cycle.; 


Integration of forecast 
rules. 

Undated Forecastme 
database. 

A new adjustable fact 
, for the next cycle. 



5 A forecast ovenide application Wcti to apply qualitative 

methods (i.e., business knowledge) to the forecastmg process. Users can change values 
generated by purely statistical methods to reflect changing business conditions. 

In one embodiment, system 1 6 maintains a naaster activity file. The master activity 
file consists of the fiill set of dimension mformation user mfdrmation, and secunty 

10 inforaiation. It also contains detailed information about all of the facts that were chosen by 
the process administrator to support the planning or adjustment process^ It' is stored in only 
one place. 

Individual uscts have their own configmation files and the appropnate fact data files. 
When the override application is started, the master data filb is read anid Hie entire dimension 
15 class is built in memory for the user. The data structure has pointers to all of the data that the 
master structure points to. A system end user can view and modify any data that he has 
peraiission to see. The data that they are allowed ^ can ibe configur^ on a user by user 
basis to ensure that they see data they are authorized to access and in a usable context. 

A lock file is created while the work file is being processed by a user to ensure atomic 
20 usage of tihe data in the file. . 

Reports 

In one embo(hment, reports are created fi-om a template. The template is like a 
blueprint for tihe report. Each time a report is created, system 10 saves the blueprint. 

In one embodiment, report templates are shared among users. Each user can then 
25 create their ovm version of the report. For example, they can run a new report, modify the 
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report .template, or '^drill" to create a different version of the report. By drilling, they are able 
to review and adjust the data anew, with niore discreet levels of detail. As an example, 
instead of simply adjusting the fact Tlanfaed Budget* up by 25% at the annual level, the end 
user may *drill down' to the mdnihly level and revifew and adjiist specific months values. 
5 ' All reports cohtam one or more c61unins and one or more rows. Each report can 

consist of multiple sections. ' if ajreport contains multiple sections, each section contains the 
same type of row iarid coiunrn^ 

The sectibn,^ column, and^ row names m^a rispdtt Veflebt the diinensions of database 20. 
These dimensions give meaning tathe- values M databasie 2^^^ therefore, important to 
10 understand how the dimensions work together. 

Every data value in database 20 is defined by one or more dimensions. To fiilly 
describe each value, a rep<)rt must contain at .least one element from each of the available 
dimensions. , . . ^ 

Period dimensions are time intervals used for identifying and consolidating the data, 
1 5 such as weekly, monthly, quarterly, and yearly intervals. Non-period dimensions describe 
other aspects of your data. Non-period dimensions may include, for example, Geo-Political, 
Product, 9nd Business Org. 

Each dimension can consist of hierarchical levels. For example, product may be a v: 
dimension. Brand and SKU ^e associated product dimension levels. The levels represent r 
20 differing degrees of detail and the paths used whai you drill. 

In one embodiment, database 20 is a data warehouse stored in one or more work files. 
Each work file can contain m unlimited number of data points. (That is, the work file size is 
hmited only by the amount of physical storage.) In one such embodiment database 20 is 
represented as a multidimensional database. Override engine 1 8 can modify and view data 
25 points at any combination of levels within the multidimensionai database md, as values are 
changed, the new values are allocated to all levels of the dimension hierarchies. This ensures 
that summary levels of the data still total correctly. User instractions to the override engine 
18 are not ^plied to the data points. Instead the instructions are stored in instruction file 30. 
Since a single instmction may change tens of thousands of rows, just storing the instractions 
30 is much faster and less expensive than storing all the changed values. Once editing is 

complete the edits can be ^plied to the work file by executing instmction application process 
34. In this way, large scale data warehouse updates can be performed offline while the user is 
working on other tasks. 
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By storing the instructions rattier than automatically sqpplying them, computer system 
10 can present the user with the results of the desired changes more quickly than in previous 
data warehouse analysis products. In addition^ by using pne or more work files rather than a 
predefined multidimensional database (MDDB) prodiuct, system , 10 can handle data 
5 warehouses which are larger in size than current database products. ^ ^ 

In one embodiment, overrijie engine 1 8 ^xtract? Jhe s^bs^^ of data entries 26 fiom 
database 20 and stores the exfract€?d da^ into cache 16. In ope such.em , 
engine 18 employs the same mephmispi used^b^^^^^ ^pJ 19 to extinct date fror^ 

database 20. For instance, in one such embpdm 91^&^^ 18 md reporting 

10 tool 19 determine the appropriate levels of d?ita to ext^ ;?:0 by c^ue^ an 

OLAP object running .on.ajstandard r^u^^^]^^ and 
reporting tool 19 look to a M identify fi^ct tebk^^^ 

determine the levels they need in each feet tabk?^ ^d to^^?^^ the da^ fixmi ^ch reqiured 
fact table. Override engine 1 8 then takes the report generated and pushes it into cache 16. 
15 Fromth^ppint on^ oyerrideeagin^ 18 i^i^^ ^ecutes th^^cpmni^ 

presents the data to the user. In addition, oyemde^^ginp \ 8 stores the conunand^in 
instraction file 30 so that if one w^t? to rerujp that repprt 1^ that has to happen is that 
the commands are re2q>plied to the loMWfi^a^^ v / , 

, As noted above, override engine 18 prpyide^ thp user or .u^ mechanism for 

20 quickly adjusting data stored in the data warel^iise dunp a subjective fact ^ju^tom^^^ 
session. InoneemlK)diinent, data stp^ed in.^ 

during such an adjustment cycle, Data that c^e out of datab 20 is ''untouched," 
Commands to alter the data are stored to instruction file 34 and are used to change the data 
being displayed to the user. The commands do npt^rhoweyer, change tjie data in either cache 

25 16 or-database 20. ; S.:.,:; -.^ v;^-^ st-- :> ■ x.i-^ ^ ^ ^ . r^? ^ - . • 

Data that' s been stored back ^tp tlie data warehouse (i.e., stored in database 20) is 
"adjusted'' or 'Svritten" data. Data in database 20 is modified using the mstmctions stored in 
instructipn file 34 via a "commit" command. In one such embodiment, data stored in 
persistent cache 16 is invalidated during execution of the "commit" command. 

30 Data which has been adjusted in cache 16 but which has not yet been \yritten to 

database 20 is in that more nebulous state in thp middle where, it is adjusted but.not 
committed. And that's the situation where cache 16 does not inatch what's in the data 
warehouse anymore. In one such embodiment, the data gets to that state by executing a 
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**rel«iie** command iii engine 18. The '^release" command causes override engine 18 to apply 
the commands not only to the data presented to the screen but also to the data within cache 1 6 
itself. In one such emlKKiiment, coinihSnds' must be •'released" before the resiilting changes 
are "committed" to database 20. Ontc a set of Commands are "^released", however, the 
original data is gone. 

In stooiher 6mibo<fiih^ erigiiiie 1 8 rnaintains a stack of old states and the user 

can Scroll back &r6u^ a li st of releases tb recover ah older state of cache 16; ' ' 

In one ^Iwidmierit, System i Orbvly finSiii datable 20 at the start of the 

forecasting pfocess! In sufeh' an ct^ 10 briJy extracts data from database 20 

at the start oif th^ fdrecs^^ 10 applies a three-step 

process: work file jgbi^^ intetactiv<rovem database storage. W 

genefatioh is aticoitipiished uSmg the fextraii^ ^hasS: fixtradt ri^ads tfie base statistical forecast 
data froiii database 20 ana stdres'ffie daSt iri a Wbifk File (cache 16)' in the file system^ 

of processor 14. ^ ^ 

Thfe remaiiiin^ ph^eis (^^^ use the data in the Work file to 

generate adjusted fact data. S^stenl 10 ^IB^s ihc user to interactively change individual 
product or rharket values to reflect charij^g bu^ess coAiiitions. Each adjustment is 
allocated tip and down the dimension luierafchy {aggregatibri lesvels). Allocations are r 
perfoimed by using the dimenaon's'dfill hii^ra^ idCTitify'lower level components of tlie 
data poirit aidjusted. These IciWdr level c6m|w^ th6n' given the new value that ' 

mairatairi^thfeir rel^ 

individual valiies so tfiat the valiies do iibt change dufmgHh^ adjxistment process (directly or 
indirebtly).''- ■ " ' ' " ''^^^ ^ ^ 

: : f Qjjce {he kdjustor has cbmpl^tfe^^^^ fbr'ah ovOTide, the adjustment is checked- 

in to the work file. The check-in process ensures that two adjustors do not adjust the same 
values aiid, in one embodiment, it allows apr6c^S adnfinikratdir to review the work before 
commit ^^ ^ ^^Mv^/.^ s ' . r ^ " ■ : . 

The final phasd of the override prcK:ess is the database storage 
Once the user or adniinistrator is satisfied with the generated overrides, the overrides must be 
committed to the database. This is Accomplished in the storage phase; ^" 

As noted above, extraction is tlie first phase of the override process. It is the 
generation of the Wbric file. Extraction builds a snapshot of database 20 and stores the 
snapshot in the Work File in the UNIX file system of processor 14. Extraction is generally 



WQ 99/57658 PCX(lIS99/09d33 

j2-_ . 

done once because it is a time-consmning process and you generally do not want pr peed a 
user to see that h^>pen. In one embodiment, the extraction process is perfonned by an agent 
executing within reporting tool 19. The agent will tjpically be.set up $o perform tt^e 
retraction process after^database 20 is refreshed : ■ / 

; In one embodiment, the extraction agent operates in response to user coinn^ds to^ 
extract the correct level of data froni <Mabase 20.^ one^suchfeOTbodixnent, one could set up 
system 10 to respond to a command such as "I want these markets by these products for this 
period on the horizon, by channel" (or whatever your o&er dimensior^^^). Awther , ^ 
command syntax might be "Give me all;my j>rod^cts,^d^ 
region level over a selwted forecast horizon" (e.g.,. cuttct^^ cuirent-plusrl2). : 

In one such embodiment, refrpsb of fte pers^ thet current flag 

moving forward one month. For exOTiple^ ifthe^cu^ swtches from Janpary to 

February, a subset of data is read frpmrdat^ase,20^^^£^^^ ^: comb^iation of facts froin, database 
20 and calculated facts formed as a fuijction of ^ stored, in cache 16. 

Any data resident in persistent cache 1 6 atythe time of the refresh ^at is unsaved or 
unconumtted is simply xyvenvritten. y . :,7 ^ [-y. : ; 

To build the Work Eile^ the system 1 Q, 3pusV4!?tOTnine which d^mrasions are drillable 
and at which aggregation levels the nonTdrillable dimensions will be forecast In one 
embodiment, this information is stored in the Category table of the; Meta4a;1^, ^ . 

r For drillable dimensions, data is stored in^^ ley^l de§ned in 

the drill hierarchy for that dimens^ion. rFor^nouiMllaW^^ dimensions„data is stored only at the 
level of aggregation specified in the Metadata, for A . , 

The Override phase allows the user to interactiyely adjust the^ base ^tistical forecast 
data. Data is read from the Work File gCTerated-by the extraction phase. The user is allowed 
to adjust values, lock and unlock values, drill^up and down] flie. drill^Me dimeaisipn's -■ 
hierarchy, as well as many other functions; / ^ ! . r ' ' 

Once an adjustor has completed a set of oyerride changes, these changes are released 
to the work file so that they are available for final commit to the warehouse. The release 
phase loads the base data from the work file, loads an instance file which contains the saved 
set of changes, and writes the new values back to the work file. 

Thexommit phase performs the commit for the Override process. First, it reads the 
Work File. .Next, the data values of the lowest-level decomposed data are read from the 
warehouse; These values are used to properly proportion the data as it is decornposed to the 
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lowe^ l^el m each dimension's hierarchy. Finally, the adjusted and decomposed data is 
writt^ to the database. 

For exaiiipie, suppose the dati in the^Woik File is stored at the regional (REG) level 
of the Market dimension. Further suppose that thfc I'owfest leVel in the Market hierarchy is 
market (MKT). Store will fekd ffie REC^^bvel;aa^ta fifbm the work jSle* decompose the FLEG 
level dati to the MKT level; ahd'fihally Simfe'the MKT level data to the database.. 



III dtie embo&meht ^at'abas^^^ a non-normklilzed.star schcfma. A 

simplified version of k sik schCTna-3£^^^ In a star sbhema, facts are 

10 stored as'data in feMieS'^^^ 

individual keys'^^df eadh diiiieri^ioli: SinitfaMyr^irii^^^^^ is stored'in dimension 

taSles^ In the '^tiodiment sliovm in Fi'g.'4J DiinTablb'40 is diinensiori table while the vi 
tables labd«l' "^^^ar^h6use PrbdUc^^^^^^^ Market'^-"Wareh6uSe Period" and - « 

"Warehouse F"ad"jffe fact tables 4i2,4^^4^^a^^^ ' 
1 5 Non-normalized star schemas are designed for very fast data aggregation md 

calculation. Such speed can be very advankgebiisih building-the subset of data to be stored 
in cache 16; Such systems cah; however;: bo^g down considerably when required to perform! 
incremental updates '(e.g^^ as is tiie base duriiSg^ foretasting session). That is where the use> 
of cache 16 arid irisiiuc^^^ 30 ai^e ihstriiniiehtial iii^ such forecasting sessions. 

20 ilie downside of astar'scheiiiai ai^pifoia^il is thai you end up with widespread 

replication of data across tables. At the' same tiihei howeveri this repHcation of data give you 
thfe ability 'to get ybiir k^y s frofti the data warehousie with Very small queries that database 20 
certairiiy ian handle quickly. So a star sthfenia approach is very well . optimized for queries 
that pull ilatfge nimibCT of rows oWof database 20; > = -■ - 

25 Standard OLAP reporting tools do not have toi undCTstand multiple levels within a 

product or indoket hierarchy of a miilti-tiered data warehoiise. This is a key difference 
between override engine 18 and a standard OliAP rerporting tool. - Override engine 18 must 
understand multiple levels at the same time because for the purposes of an adjustment, there 
may be interdependencies on the levels themselves for a reporting tool that essentially say, 
30 "O.Ki^ here's datji at this level," or ••Here's data' at tfie combination of these two levels." 

They're largely iiidependent of each other and you can concentrate on one or the other at any 
given time: Because of that, override engine 1 8 does not run directly against the warehouse. 
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Instead, override engine extracts that data, performs calculations where necessary and stores 
themodifieddata tocache 16. > r,^ ' 

In order to operate correctly, however, with datal)ase 20, override engine 18 and 
instruction application process 34 must underst?md the 20. In one 

embodiment, as is shown in Figure 3, override engme 18,^a^ q[)plication process 

34 extract the structure of database 20 by reyipwing the consents of Metadata table 36. 

Such an approach permits the Uise pf an unliimt^ an easily alterable number of 
dimensions. In its simplest form, metMa^,t?ible 36^ 40 and 

a fact table 46. In one embodiment, each diromsion^^^^ H^y, uniq^ie . 

description field, a leyel column and a hierarchy l€?yeI,CQlurnn.^ addition, pOTod dini 
table includes a unique sequience nxraibCT coJ year coliimn and a 

current period qohmm, Fact tab^e 4^ ii^^^ whi|ch a^^ iji^entical in type structure 

to thekeys Usted-indimpnsi^^ ^ >1 >rf? r n- ^ 

For example, the metadata table can be used |adrive a drill hierarchy th^t tells 
database 20 that days make up weeks and >yeete make^iip montte and months make up 
quarters, quaitep make up halves, halye^^ Or that four quartesrs make up 

year, so that the user can jvmip and skip things. .Same, t^ in the product hierarchy. 
Override engine 18 and r^>orting engine 19 imderstand the Metadata structure and use the 
data stored in the dimension tables to extract data firom database 20. , , 

( : For example, a certain table .may jnclude a category of sales l>y total U.S. And maybe 
it is category, manufacturer, brand. And thf^ pyer in another t^^^ are lists of SKUs by store. 

Another approaches to haye a separate, facttaljle for each month's data. TTiat way 
when the next month's data arrive,, it gets placedjinto a ney fact table and stored in database 
20. An advantage of such an ^proach is that if one of the^^t tables gets, lost or corrupted, 
you can reload that month. In additioii, as 4^ Wgger and bigger, its value to the 

company increases jmd .so does the c^st of jnaintaimng the database. Relational databases can 
scale into niuch larger data sizes and.are much more maintainable than a corresponding multi- 
dimensional database. , , . . - J . 

It should, be understood that database 2p may be distributed across a number of 
computers. In one anbodiment, such as is_ shown in Fig. 5, computer system ,50 includes a 
server;52 connected to a processor 58 and a processor 62, by a network 56. - In addition, server 
52 is connected to a plurality 1 through N of workstations ,62. Processor 58 and processor 62 
store portions of database 20 in memories 60 and 64, respectively. 
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in one'^emtx>dimen as is shown in Fig. 5^ server 52 maintains a separate instantiation 
54 of override engine 1 8, reporting tool 19 and cache 16 for each user performing the 
forecasting fimctiori. Li another embodiment (riot shown), a single cache 16 is used across all 
users, hi either case, changes to cache *1 6 Wniafintairied in a' separate instruction file 30 for 

5 each inktaiitiation 52 and are' only ap^pliea to the baiche on receipt of a "releaise" command. 

K one embodmiejnt, procesk)r 58 is% multiipie processed such as the Hewlett 

Packard HP9do6 runhirig aii O^ application whiie processor is a Tandem 

Himalaya 128: In one Wiich eimbbidiimS^t^ aij^g^^ stored in the Oracle database on 

the HP machine while the 1^ in the Himalaya machine. The Metadata 

10 tells tti^ program the apprdpnate pfbc^iS^df to whibfi a query shoiald be adSressbci; For ^ 
example^ if dat^ is 's^tordil by catego^'by sfdre^ 18 can 

deterinii^ this bt? looking at tfie Mkkt^t£ ^f;'6ri'dbieyfheThifid^ storing iaggregate 

data in cache 16, it v^dll determine from the Metadata t^ble thai it shdvild extr^t such data 
from the Oracle daitabase oil pfbbessd^^ ' ■ ^ j--- . * 

1 5 in aiibther' einbodiinerit^ wfemdS Vn 18 and rdpbrtin^ tool 19 run as a relatively - : 

thin windows cli^t that essentially jUist ptoX^ 32:4A:li the a^ all the storage, 

and all the rieial processing is done on UNIX serve? 52i. i 7 F 

Locking Values in the Baftahdse ^ ^ ' - ' ' ^ — ' ^"^ 
In one embodiment, overri 

20 As noted abbve, 6n i releai&e, the bliahgei^ at orfe lei^el bf the hierarchy are pushed down to 
each 'of the'sublevels of the'hidrarchy. ^or^exain^le,if p^^ is increased across the 

board by 10,006 units, the 1 6,000 liiiits^ air^ distribifited prbpbrtionally across each of the 
entities at the lowest iervel of i-i' < ;i > 

In certain situations, sucK aii approach idoes^n^ ^ehse. For instance, if one of 

25 three manufabturing plants is operating at capacrty,^it Biakes little seiise to distribute the 

increase prbportiohally to the plant operating at edacity. For such situations, bveiTide engine 
1 8 includes a locking mechanism which can be used to lock the output of any oif the 
manufacturing plants to a Certain value. Any increases are then distributed proportionally 
across the remaining, unlocked, manufacturing pl^ts. • * ; 

30 In one embodiment, the adjustment report displays the values of the dimensions the 

user selects on a Template dialog SCTeeh/ The on^^ 
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forecast are found in the unshaded cells. Ix)cked values are shaded red and non-adjust^le 
values are shaded yellow. > ? ^ 

The user can adjust a value at any level and the effects ripple through the rest of the 
product hierarchy. For example, if he or she adjusts a SKy.Vjalue, the values of products at 
higher levels change to account for the lower level adjustpaent.. p^^ if the user adjusts a higher 
level valucy the values for the lowCT level pro^ ^ ^ ^ r 

Likewise, when ,the user locks a value, pipduQt yaluips both up and do:wn the hierarchy 
may be affected. ForexampHi^the^Joiil^^ 

llie diagram shown in Fig. ^.^ an exa^ipteof dec^ 
based on current forecast data. Jl^e current ^djTistment d^ta is r^tri^y^d a$ p^ufacture level 
70 and, after an adjustment, is proportionally prpp^^ 

If, hoAveyer, the manufacture prq^ 70 and one Brand 

product value was adjusted from 50 Jo 60^ Ae j^justora^^ effects are sliown in Fig. 6b. 

As noted above,,in pne: encAodime^ tool 19 and override engijie 18 

have the ability to linut the effect.of coi^ frp™ database 20 

does not include the entire hierarchy for -all dimensions. For instance, one may care about 
multiple levels of product dimension, but woi^ld hke to limit the.iiriarke^ ad]ust^lents to a 
particular level (e.g., make the market adju^tmCTts at a to^^ ^ S- )^, .In one ^uch embodiment, 
the user can select the lev^el at which data is a^djus]ted., ?or example, pne would tell ovenidp 
engine 1 8 to, 'TVIake your product <hmer^ion <^^^ market cjimension non- 

drillable, make your period dimension non-drijjl^le,*' Adji^^tmpts then must only be driven 
down the product hierarchy for display; the ovenride engine ^pes not h£ive,to try to drive it 
down the other hierarchies. As you add multiple <^ 

that has to happen when you make adjustments expands exponentially. Not only does it go 
down product, but it has to go do>yn product for every maricet, or every subnciarket, or every 
submarket of every submarket and you end up with a huge matrix of numbers to keep track 

of . ... - V - ' . - - ^ . - : ' - ; 

E)rilling allows the user to display the fact values at different product levels, both up 
and down the.product hierarchy. With drilhng, one can see.the aggregate values of a group of 
products; or can identify the specific values that went into the aggregate ampunt. 
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Diiiiing up works with any product where there are other products at a higher level. 
For example, one can drill up on a value if he or she is at SKU level 74 shown in Fig. 6a. 

Finally, a user can drill down with any product if he or she is at a level above the 
lowest level, SKU. For example,' one can drill xlown if at itiianufacture level 70 in Fig. 6a. 

If^ for instance, yoii want t6 adjust the^ iimbuht of a particular soft drink sold in a 
particular city, you could dther maJce iHk i dimension. You 

can do it in one of two ways. dan set ySirf inM:et dimension as drillable, and then 

poll the whole riiaricdt iaerarc If, 
however, you did not need that flexibility in drilling and the associated performance impact 
of allocating the'charigds throii;^ tiiat dim'ensibiiV the tnaarket dimensibh 

*non-cirillabie* mid brily iee datd St th^^ei^plei^d^* t6^ can jifet^i^ whatever city you 
warit^ and make ybiir aidjustk some 
level below city, once you do a "commit" it is going to drive the mmibers to the bottom of all 
the hifer^chies. (Iii one dnt>odiAitnti bV^d'e' imgihb tirives ffie numbers down to the bottom 
of all the biCTarchi^s by modifying eaeh MiiA^ a^ a pisrcent of contribution to the total.) 

' Fbi- eiiriple, if gi vOT cities withiii those regions 

and stores witMh &ose cities, you were gpfi^^ levels you only : 

need t6 ihairitain dati in cache 16 at that Ifevel:^ td d6 this, override engine 18 initiates a 4- 
transfer of d:ka at the itb^riic IfeV^^ and piishes it up to the regional level. ' -^ 

The aggif^egated regional data is then Stored iii cabhe lis smd commands from the user are 
applied kgaihsVthat aggrega^^ Thra;^wii^ you iire read^ 

"commit" conurianiJ is exwuted aii^'the ihbdt^ to the atomic level in 

database 10. The result is that usfer cbmmands^et execu^^^ at the regional level and 

get stSbreiii accurately at the at^^ ^ - :3;o : . : ; ; . - . : 

* * Fot eixainpl^, bnfe might de^ if volume increased by 10% in 

a regibri suLch as'the Great Lakes temtoi^: llie-chahge would be made at t^^ level 
andi, when committed, it is distributed to the cities Sis' a fiuiction of the percentage they 
contribute to the rejgiohal total. ' ■ ' * ' 

No matter the level where the adjustment is made, once the warehouse is updated, it is 
updatbd at the lowest atomic level and then in this Case for reporting engine 19, then it can 
automaticailly pick up a report of it baseid oh market hierarchy, period hierarchy, etc: All that 
is a trade-off bcitween speed and flexibility. You can go ultimately flexible if you've got 
enough iron behind the thing to drive it. And if you do not, then you can cut back. 
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Also from a security pCTspective, quitQ often if it is sales people that are using it for 
the adjustments, they do not want them to see other territories, for instance, other people's 
clients, other sales guys* clients. Otherwise they, start playin that you do not want 

them playing. It is; a combination of those things, , , v 

Override engine 18 understands all of reporting tool 19's filters and calculated facts; 
it also understands the hierarchies of system IQ.. Essjmtiany,evepr^^ in the Metadata that 
can be defined, is read by reporting^ool 19 ajid^o^^enid^^ 1 8 and Aey AyiU V(?p *ose 

definition.' So you can create a cjdculatedfictjm rTO^ pvenride en^^ 18 

wiU:aiUomatieally see it andknowjiowjto.^ - r : -n . : v 

. (An ex^ple of a-bsdculated facl WGU^ is calculated 

as units times price.i You would not, ti>^^fqi;e store gcp^sj:eyp>u^^^^ you calculate it 

off of the two.that you did store {pricjetand.iuiits)^^ fact is forecpt error. 

Facts calculated as a function of an actual fact and e,forecaste^^ degiyed, not stored. 

Override engine l S automatic^ly reads tfie defiindtif n pf each c^qulat^^^ out of the 
Metadata and sqpplies them on the fly to the data read j{i;oiti databas^e 20.) _ 

Mn one embodiment^ forecasts arcr stor^ database 20. For 

instance, one could include a dimension labf^led "|cenario ■ ' into top/warphoTusp, It can, 
therefore, be critical to have the ability tp add exfa^ d™ensions to systems 10 and 50. 

As was noted in connection with Figure 2, in one embodiment cache 16 is a persistent 
cache stored as a B-tree in a UNIX file system on processor 14. The B-tree allows you to 
essentially go with a much more compact data storage in situations where the matrix of data 
is populated sparsely (such as in, for example, customer centric databases) yet, at the same 
time, a B-tree implementation does allow you to get reasonably fast access times. 

In one such embodiment, the persistent cache is implemented using standard Rogue 
Wave B-tree code. The performance of the Rogue Wave B-tree code can be enhanced by 
replacing the standard I/O stream (which is Avrite character by character) with an I/O stream 
which writes data in large blocks of data. 

In addition, with a B-tree implementation, one can trade off size for speed by 
increasing or decreasing the number of branches and subbranches in the tree. That is, the 
deeper you make the B-tree, the slower yoxur access time but, at the same time, the smaller the 
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actual physic^ storage. In one embodiment, the structure of the B-tree is optimized for size 
over speed. 

In niore densely pabkeii matriciss of flata, cdche 1 6 can be implemented as a set of file- 
based arrays. Such arrays are very fast, reqxiirinrg only simple calculations to get to the right 

5 locations. They c^ be, hov^^ver; veiry'iiK?fficieht space-wise'. ^ Typically, if one of the 

diinensions of the (lata war^ho^^e is custdinCT; aiid the nunib'er of potential customfcrs is large, 
you'ai^bettCT off selecting a B-tt^^^^ ' 

Override eh^ne 1 8 alloWs the u^er'ti^'^b slices of data firom a database that may be 
too big to comprehend in its entirety. Override 'OTgine'l 8 klldws you td sUce a portion of the 

1 0 daltabase out, which sometimes iri^ ydu^'th^Mrt^fer ta as a datainark, automaticaUy 
aifd intduiOTtly tlif^ii^ be dealt with either 

withm or oiitsid^ thJ datibase qiiieklj^i^ 

override eiigink iS kutoirtatically ahd'Mft pushed ^etn ^ack info database 20. The t 
result is ^ datkbase' t6<il Wliicli eiimiriafe »jlg'^qpiei^ fr^ffib bottlm^kt^ approaches 

15 to relational database maii^gemeiitsfysteinfe'/^ ^ 
Althou^ specific embddiments haVb'been illuStKit herein, it will be 

appreciated by thosfe of ordinary skill in thFaitttat aiiy a^^ which is calculated tor 

achieve the same purpdse iiiay be'substitirted' tot Hvt sjiecific embcfdiment shown. This , a 
application is intended to cover any adaptations or variations of the present invention. 

20 Therefore, it is intended that this invention be Umited only by the claims and the equivalents 
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What is claimed is: 

1 . A data warehouse, comprising: 

memory, wherein the memory incliides a database haying a plurality of data entries; 
a processor connected to the memory, wherein the, p mcludes: 

. r,,. a cache;^and^ . ^r,o r ■ . .^^ 

an override engine; 

wherein the cache includes a subset of; the jhii^ entries apd wherein the 

override engine extracts data from the cache for yie\wn^^ user,^ modifies the data in 
response to one or inore user commands and .sayes the user commands to a file for later 
application to the database. f r . 

2. A data warehouse, comprising: . , r . - . 

memory, wherein thg meniory mclu^^es,^ plinulity of data entries; 

a server connected to the memory, wherein the processor includes: 

acache^and --.^ ^ , v^..rs.,t^ /t.fr;--^' n\ h:v-.c; -~ ■ ^^u-^ - v- ■ ■ -^ :, 

an override engine; and : - - . - » v 

a client workstation connected to the.sCTyer;. , 

wherein the server operates in response to user commands to store a subset of the 
plurality of data entries in the cache, wherein the pyemde engin^ extracte data from the cache 
and sends the data extracted fix)m the cache to , tiie^chCTt wor^ for viewing by a user, 
modifies the data in response to one or more of the;.uspr cpnimands, saves the user commands 
to a file and operates in response to- a cpnimit command to modifies the databasjp based on the 
iiser commands stored in the file. a - > v - r; y." ^ 

3. A method of reporting data from a data t\yarehouse,rthe method comprising the steps 
of: 

, providing a server and a memory ideyice; 

storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extracting a subset of data entries from the database; 
: V storing the subset of data entries on the server, ; . 

modifying the data entries stored on the server in response to user commands; 
reading data from the modified data entries stored on the server; and 
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displaying the data to the user. 

4. A method of forecasting, comprising^ 
providing a seiver md a meiiiory de^^ 

storing a database in the memory device, wherein the database includes a plurality of 
data entries; 

extociting a^ siitset of d^^ 
storing the Vtibi'^t of dati 

mbdifyiiig th6'<iat¥ ^iitries siKrbii 6^ tfie server in -reisiionse to usei commands; 
storing the user commands; 

reading data from the modified data entries stored on the server; 
displaying the data to the user; and ^ ' ' ^ - ' ' ^ 

hiodiif^TOg the^d^^^ b- v 

5. A method of increasing the speed in which changes to a relational database are 
reflected back to the user, comprising the steps of: ' ; v 

extracting a subset of data from the ifatabase, whereiii flie stfep of extracting includes* 
the st€^)s of: " '■' " '^--^ ' ■ " ' — - - 

^ '-^ 

stomgffiesubisiet^fd^ ^ 
* receiving' a data modificatibtf cbhi^^ - '^ 

storing the daita modification CO ' 
applying the data modification command against the subset of data stored in the 
cache, wherein the step of applying the data modification command includes the step of 
modifying the siibset of data tb-reftect ippHcatib command. 

6. The method according to claim 5, wherein the jstep of ^plying the data modification 
command further includes the step of modifying the representation of the subset of data to 
reflect application of the command. 

7. The method according to claim 6; wherein the step of applying the conimand further 
includes* the steps of: ■ ^ 

waiting for a "commit" coiniriand; and 
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on receipt of the "conunit" command, modifying data in the database to reflect 
application of the data modification command. 

8. The method according to claiiA 5, wherein jthe step of applying the conunand further 
inchides the steps of: ] _ [ 

waiting for a "commit" command; and ; 

on receipt of the "commif ' coimnand^! modifying data in th^ database to reflect 
application of the data modification command. , ' - > ■ ; 



9. A data structure for updating a database, comprismg: 

a plurality of user data-modification commands, wherein the plurahty of user data- 
modification commands provide information for mc(difying information in a database (20). 

I 

10. The data structure of claim 8|whCTein: 

the plurah^. of user data-moclification comi^iands provide data-modification 

information for plication to aggregated regional data stored in a cache (16), and 
> modification information that gets drilled down to atomic level in the database (20). 
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